Terminal apparatus and associated methodology for automated scroll based on moving speed

ABSTRACT

To improve ease of operation in a case where a display device and a small terminal apparatus are connected to each other, and coordinated display is performed. When a contact or approach of an object having a size greater than or equal to a threshold value with or toward the surface of a display panel is detected, a video or an instruction corresponding to the motion detected by the motion sensor is output. The display device, on the basis of the video or the instruction that is output, displays a video in which the position of a pointer or the like is changed in response to the motion detected by a motion sensor.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/900,586, filed on May 23, 2013, which claims the benefit of U.S.Provisional patent application 61/651,606, filed in the USPTO on May 25,2012, the entire contents of both of which are incorporated herein byreferenced.

TECHNICAL FIELD

The present disclosure relates to a terminal apparatus including adisplay unit, a display system in which the terminal apparatus and adisplay device are combined, a display method that is applied to theterminal apparatus, and a recording medium.

BACKGROUND ART

In recent years, mobile phone terminal devices having highfunctionality, which are called smartphones, have become popular.Smartphones, which include a display panel having a comparatively highresolution, can display moving images and still images. It is possiblefor the display panel to display, for example, a video captured using acamera function incorporated into the smartphone, with a highresolution. Alternatively, a smartphone accesses the Internet, making itpossible for the display panel to display a video obtained from varioushome pages.

Furthermore, some smartphones include a video output unit. Some videooutput units provided in smartphones include a terminal for an interfaceof, for example, the HDMI (High-Definition Multimedia Interface)standard. A terminal of the HDMI standard is a terminal from which highresolution video data, such as, for example, horizontal 1920pixels×vertical 1080 pixels, which are the numbers of pixels of onescreen, is output.

Such a video output terminal of the HDMI standard for a smartphone isconnected to the video input terminal of a display device, such as atelevision receiver or a computer screen, by using a transmission cableof the HDMI standard. Thus, it is possible for the display device toexpand and display the same video as the video being displayed on thedisplay panel of the smartphone itself.

Furthermore, it has been proposed that a smartphone includes a wirelessvideo output unit, and a video is transmitted to a display device bywireless transmission. That is, a smartphone has incorporated thereintoa short-distance wireless communication function that performs wirelesscommunication with adjacent devices. By using this short-distancewireless communication function, it is possible to transmit a video to adisplay device wirelessly.

PTL 1 discloses a mobile phone terminal device including a terminal ofthe HDMI standard.

CITATION LIST Patent Literature

[PTL 1] Japanese Unexamined Patent Application Publication No.2009-10587

SUMMARY

Smartphones are designed so that various operations can be performedusing the touch panel. For example, when a home page obtained by thesmartphone by accessing the Internet is displayed, an operation ofinputting characters in accordance with the home page being displayed oran operation of moving to another associated screen, is performed by auser by performing a touch operation on the display screen.

Even in the case where the video output by the smartphone in the mannerdescribed above is being displayed by another display device, it ispossible for the user to perform various operations associated with thedisplay by performing touch operations on the display screen of thesmartphone. However, touch operations of touching the display screen ofthe smartphone have a problem in that operation is not always easy toperform depending on the manner in which the operation is to beperformed. For example, in a state in which the video output by thesmartphone is expanded and displayed on another display device, it isnecessary that the user performs a touch operation of touching thedisplay screen of the smartphone while viewing the video that isexpanded and displayed on the display device. For this reason, it isnecessary for the user to perform an operation while alternately viewingthe video expanded and displayed on the display device and the video ofthe comparatively small screen of the smartphone. Thus, performing atouch operation does not lead to improved ease of operation.

The inventors have recognized the necessity of improving the ease withwhich an operation is performed in a case where an operation related tothe display video when the video output by the terminal apparatus is tobe displayed on another display device.

A terminal apparatus according to the present disclosure includes adisplay panel arranged on a housing; a display processing unit thatprocesses a video displayed on the display panel; an output unit; atouch-panel unit that detects a contact or approach of an object with ortoward a surface of the display panel; a motion sensor; and a controlunit.

The output unit externally outputs a video processed by the displayprocessing unit or an instruction regarding a video.

The control unit performs control such that when the touch-panel unitdetects the contact or approach of the object having a size greater thanor equal to a threshold value, the output unit outputs a video or aninstruction corresponding to the motion detected by the motion sensor.

A display system according to the present disclosure includes a terminalapparatus and a display device.

The terminal apparatus includes a display panel arranged on a housing;

a display processing unit that processes a video displayed on thedisplay panel;

an output unit that outputs a video processed by the display processingunit;

a touch-panel unit;

a motion sensor; and

a control unit. The control unit performs control corresponding to themotion detected by the motion sensor when the touch-panel unit detects acontact or approach of an object of a size greater than or equal to athreshold value.

The display device displays a video output by the output unit of theterminal apparatus. Then, under the control of the control unit of theterminal apparatus, a display device displays the video that is changedin response to the motion detected by the motion sensor.

A display method according to the present disclosure performs a displayprocess of displaying a video on a display panel, an output process ofoutputting the video from an output unit, and a control process. Thecontrol process causes the output unit to output a video or aninstruction corresponding to the motion detected by the motion sensorwhen a contact or approach of an object of a size greater than or equalto a threshold value with or toward the surface of the display panel isdetected.

A recording medium according to the present disclosure is a recordingmedium having recorded thereon a program for causing a computer toexecute a program, and includes

a display process of displaying a video on a display panel;

an output process of outputting a video from an output unit; and

a control process of controlling output from the output unit. When thecontact or approach of an object of a size greater than or equal to thethreshold value with or toward the surface of the display panel isdetected, the control procedure causes the output unit to output a videoor an instruction corresponding to the motion detected by a motionsensor.

According to the present disclosure, in the terminal apparatus, when thecontact or approach of an object of a size greater than or equal to thethreshold value to the surface of the display panel is detected, thevideo or the instruction corresponding to the motion detected by themotion sensor is output. For this reason, it becomes possible for theuser to perform control regarding the video being output by performing amotion, such as swinging the terminal apparatus in a state in which theuser holds the terminal apparatus in such a manner as to cover thesurface of the display panel. Furthermore, by only changing the way ofholding the terminal apparatus so as to hold the terminal apparatus insuch a manner as to cover the surface of the display panel, the terminalapparatus is automatically switched to a state in which control can beperformed by such motion, providing the advantages that the user doesnot need to perform a switching operation.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of aterminal apparatus and a display device according to a first embodimentof the present disclosure.

FIG. 2 is a diagram illustrating an example of the outer shape accordingto the first embodiment of the present disclosure.

FIGS. 3A and 3B are illustrations illustrating an operation exampleaccording to the first embodiment of the present disclosure.

FIG. 4 is a flowchart illustrating a control example according to thefirst embodiment of the present disclosure.

FIG. 5 illustrates the flow of the control state according to the firstembodiment of the present disclosure.

FIG. 6 illustrates another example (example of changing output video) ofthe flow of the control state according to the first embodiment of thepresent disclosure.

FIG. 7 is a flowchart illustrating a processing example for eachoperation according to the first embodiment of the present disclosure.

FIGS. 8A-8C illustrate change examples of operations and display videosaccording to the first embodiment of the present disclosure.

FIGS. 9A and 9B illustrate other change examples of operations anddisplay videos according to the first embodiment of the presentdisclosure.

FIGS. 10A and 10B are illustrations illustrating a tap operation andscroll operations according to the first embodiment of the presentdisclosure.

FIG. 11 is a flowchart illustrating a control example according to asecond embodiment of the present disclosure.

FIGS. 12A and 12B are illustrations illustrating operation examplesaccording to the second embodiment of the present disclosure.

FIGS. 13A and 13B are illustrations illustrating operation statesaccording to a modification of the embodiment of the present disclosure.

DESCRIPTION OF EMBODIMENTS

Examples of a terminal apparatus, a display system, a display method,and a recording medium according to embodiments of the presentdisclosure will be described below in the following order with referenceto the drawings.

-   1. First Embodiment-   1-1. Configuration of terminal apparatus (FIG. 1, FIG. 2)-   1-2. Examples of operation states (FIG. 3)-   1-3. Control process based on operation (FIGS. 4 to 7)-   1-4. Examples of specific operations and displays (FIGS. 8 to 10)-   2. Second Embodiment-   2-1. Control process based operation (FIG. 11)-   2-2. Examples of specific operations (FIG. 12)-   3. Modifications (FIG. 13).    <1. First Embodiment>    [1-1. Configuration of Terminal Apparatus]

FIG. 1 illustrates a configuration of a mobile phone terminal device 100and a display device 200 of the present disclosure.

The mobile phone terminal device 100 is a highly functional terminaldevice, which will be referred to as, for example, a smartphone. Thedisplay device 200 is, for example, a television receiver.

The mobile phone terminal device 100 includes an antenna 101 forperforming wireless communication with a base station for wirelesstelephone. The antenna 101 is connected to a wireless communicationprocessing unit 102. The wireless communication processing unit 102,under the control of the control unit 110, performs processes oftransmitting and receiving a wireless signal. The control unit 110transmits a control instruction to the wireless communication processingunit 102 through a control line CL. The control unit 110 reads a program(software) stored in a memory 150 through the control line CL andexecutes the program, thereby controlling each unit of the mobile phoneterminal device 100. In the memory 150 included in the mobile phoneterminal device 100, data that is prepared in advance, such as aprogram, is stored and also, data generated by user operation is stored.In the memory 150, storage and reading of data are performed under thecontrol of the control unit 110.

During voice phone call, voice data for phone call, which is received bythe wireless communication processing unit 102, is supplied to an audioprocessing unit 103 through a data line DL. The audio processing unit103 performs a demodulation process on the supplied voice data so as toobtain an analog audio signal. The analog audio signal obtained by theaudio processing unit 103 is supplied to a speaker 104, whereby audio isoutput from the speaker 104.

Furthermore, during the voice phone call, the audio processing unit 103converts the audio signal output by the microphone 105 into voice dataof a transmission format. Then, the voice data converted by the audioprocessing unit 103 is supplied to the wireless communication processingunit 102 through the data line DL. Furthermore, the voice data suppliedto the wireless communication processing unit 102 is packetized andtransmitted in a wireless manner.

In a case where data communication or transmission/reception of mail viaa network, such as the Internet, is to be performed, the wirelesscommunication processing unit 102 performs transmission and receptionprocesses under the control of the control unit 110. For example, thedata received by the wireless communication processing unit 102 isstored in the memory 150, and a process, such as display, based onstored data is performed under the control of the control unit 110.Furthermore, the data stored in the memory 150 is supplied to thewireless communication processing unit 102, whereby the data iswirelessly transmitted. In a case where the data of the received mail isto be discarded, the control unit 110 deletes the data stored in thememory 150.

The mobile phone terminal device 100 includes a display processing unit120. The display processing unit 120, under the control of the controlunit 110, displays video and various information on a display panel 121.Furthermore, regarding video to be output to the outside, the displayprocessing unit 120 generates it under the control of the control unit110. For the display panel 121, for example, a liquid-crystal displaypanel or an organic EL (Electro-Luminescence) display panel is used.

Furthermore, the mobile phone terminal device 100 includes a touch-panelunit 130. When the surface of the display panel 121 is touched by anobject, such as a finger or a pen, the touch-panel unit 130 detects thetouch position. For the touch-panel unit 130, for example, anelectrostatic capacitance type touch panel is used. An electrostaticcapacitance type touch panel can detect a state in which a finger or thelike directly touches the surface of the display panel 121 and also, candetect a state in which a finger or the like approaches the surface ofthe display panel 121. A case where touch detection is mentioned in thefollowing description may be either a case in which the surface of thedisplay panel 121 is directly touched and an operation is performed, ora case in which an operation is performed as a result of a fingerapproaching the surface of the display panel 121.

The data on the touch position detected by the touch-panel unit 130 istransmitted to the control unit 110. The control unit 110 executes theapplication to be started on the basis of the supplied touch position.

The mobile phone terminal device 100 includes operation keys 141 to 145.The operation information of the operation keys 141 to 145 istransmitted to the control unit 110. An example of specific arrangementstates of the operation keys 141 to 145 will be described later.

Furthermore, the mobile phone terminal device 100 includes ashort-distance wireless communication processing unit 107 to which anantenna 106 is connected. The short-distance wireless communicationprocessing unit 107 performs short-distance wireless communication withadjacent terminal devices and an access point. The short-distancewireless communication processing unit 107 performs wirelesscommunication with another party in a range of, for example,approximately several tens of meters by applying a wireless LAN (LocalArea Network) method defined by, for example, the IEEE 802.11 standard.

The mobile phone terminal device 100 of the present disclosure cantransmit the video being displayed on the display panel 121 by thedisplay processing unit 120 to an external display device in thewireless communication in which the short-distance wirelesscommunication processing unit 107 is used. Furthermore, the mobile phoneterminal device 100 can transmit various instructions and controlsignals, which are associated with the video, in the wirelesscommunication in which the short-distance wireless communicationprocessing unit 107 is used. As described above, the short-distancewireless communication processing unit 107 functions as an output unitthat outputs video and instructions to the display device throughwireless transmission.

In addition, the mobile phone terminal device 100 includes a motionsensor unit 108. The motion sensor unit 108 judges the state of themobile phone terminal device 100 on the basis of the detection data of agyro unit 108 a and an acceleration sensor unit 108 b. The gyro unit 108a detects the magnitudes of angular velocities of three directions:vertical, horizontal, height. The acceleration sensor unit 108 b detectsthe magnitudes of accelerations of three directions: vertical,horizontal, height. The angular velocities and accelerations that aredetected by these sensor units are angular velocities and accelerationsthat are applied to the housing forming the mobile phone terminal device100.

The data on the state of the mobile phone terminal device 100, which isdetected by the motion sensor unit 108, is supplied to the control unit110.

Furthermore, the mobile phone terminal device 100 includes aninput/output processing unit 160. A terminal part 170 is connected tothe input/output processing unit 160. The input/output processing unit160 performs data input and output processes with the outside through atransmission cable 300 connected to the terminal part 170. The terminalpart 170 is, for example, an output terminal for the interface of theHDMI (High-Definition Multimedia Interface) standard. The terminal part170, which is an output terminal for the interface of the HDMI standard,is a terminal that outputs, to the connected transmission cable, highresolution video data, such as, for example, 1920 horizontal pixels×1080vertical pixels as the number of pixels of one screen, and audio dataaccompanying the video data. The video data output by the terminal part170 is, for example, created by the display processing unit 120.Furthermore, the terminal part 170 can transmit and receive a controlinstruction or the like by using a CEC (Consumer Electronics Control)line defined by the HDMI standard.

Next, the configuration of the display device 200 shown in FIG. 1 willbe described.

The display device 200 is configured as, for example, a televisionreceiver. The display device 200 includes a terminal part 201. Theterminal part 201 is made to be, for example, an input terminal for theinterface of the HDMI standard. The video data obtained at this terminalpart 201 is supplied to the switching unit 202. The switching unit 202selects any one of the video data output by the terminal part 201, thevideo data output by a tuner 204, and the video data output by theshort-distance wireless communication processing unit 206. The selectionin this switching unit 202 is performed under the control of the controlunit 203. The tuner 204 is a tuner that receives broadcast waves oftelevision broadcasts or the like so as to obtain video data. Thedisplay device 200 may include, in place of the tuner 204, a receivingunit that obtains moving images and still images through the Internet.

The short-distance wireless communication processing unit 206 performsshort-distance wireless communication with adjacent terminal apparatusesand an access point. In order that this short-distance wirelesscommunication is performed, an antenna 205 is connected to theshort-distance wireless communication processing unit 206. Theshort-distance wireless communication processing unit 206 performscommunication in the same wireless communication scheme (for example,the wireless LAN scheme) as that of the short-distance wirelesscommunication processing unit 107 of the mobile phone terminal device100.

Then, the video data selected by the switching unit 202 is supplied tothe display unit 210. The display unit 210 includes a large displaypanel 211 when compared with the display panel 121, and performs aprocess for displaying a video on the display panel 211 on the basis ofthe video data. At the time of operation description (to be describedlater), the switching unit 202 of the display device 200 is in a statein which the video received by the short-distance wireless communicationprocessing unit 206 is selected.

FIG. 2 illustrates an example of the shape of the mobile phone terminaldevice 100.

The mobile phone terminal device 100 configured as a smartphone isarranged with a display panel 121 on the surface of a portrait housing.The display panel 121 has, for example, a length of a diagonal line ofapproximately 10 centimeters. Display on this display panel 121 isperformed by the driving by the display processing unit 120.Furthermore, the touching on the surface of the display panel 121 with afinger or the like is detected by the touch-panel unit 130.

Furthermore, the mobile phone terminal device 100 is arranged with threeoperation keys 141 to 143 on the surface of the portrait housing in thelower side of the display panel 121. On the side surface of the mobilephone terminal device 100, an operation key 144 serving as an up key,and an operation key 145 serving as a down key are arranged.

In addition, a terminal part 170, which is an output terminal for theinterface of the HDMI standard, is arranged in the upper end portion ofthe mobile phone terminal device 100. This terminal part 170 includes aterminal cover, and the terminal cover is uncovered when a cable isconnected.

[1-2. Examples of Operation States]

A description will be given of the overview of an operation state inwhich the short-distance wireless communication processing unit 107 ofthe mobile phone terminal device 100 and the short-distance wirelesscommunication processing unit 206 of the display device 200 arewirelessly connected to each other, and the display device 200 displaysa video output from the mobile phone terminal device 100.

FIGS. 3A and 3B illustrate operation examples in a case where the mobilephone terminal device 100 of the present disclosure is arranged in thevicinity of the display device 200, and the display device 200 displaysa video output from the mobile phone terminal device 100.

FIGS. 3A and 3B illustrate an example in which, while holding the mobilephone terminal device 100 with one hand h, the user touches the surfaceof the display panel 121 of the mobile phone terminal device 100 byusing the thumb f1, thereby performing a touch operation. In thisexample, the fingers f2 to f5 other than the thumb f1 are positioned onthe rear side of the mobile phone terminal device 100, and the userholds the mobile phone terminal device 100 by using these fingers f2 tof5 and part of the palm of the hand.

Then, for example, the display panel 211 of the display device 200displays the video output from the mobile phone terminal device 100. Thevideo to be displayed by the display panel 211 of the display device 200may be either the same video as a video displayed by the display panel121 of the mobile phone terminal device 100, or a dedicated video, whichis generated so that the video is displayed on the display device 200.

At this time, in a case where there was a touch operation on the surfaceof the display panel 121, the video displayed by the display panel 211of the display device 200 is changed to a video that reflects the touchoperation. For example, the display position of the pointer (cursor) inthe video displayed by the display panel 211 of the display device 200is changed in response to the touch position that is detected by thetouch-panel unit 130 of the mobile phone terminal device 100. Theoperation of the position of the pointer is an example, and it ispossible for the user to perform operations regarding various displayvideo with the touch operation of the display panel 121. In theoperation state shown in FIG. 3A, only the thumb f1 is positioned on thedisplay panel 121 of the mobile phone terminal device 100, and most ofthe surface of the display panel 121 is not covered with fingers or thelike.

Then, the mobile phone terminal device 100 of the present disclosure ischanged from the state shown in FIG. 3A to a state in which theoperation by the motion of the mobile phone terminal device 100 itselfis possible by holding the mobile phone terminal device 100 by the userso that the palm of the hand is positioned on the obverse side of themobile phone terminal device 100. FIG. 3B illustrates a state in whichthe user holds the mobile phone terminal device 100 at this time.

That is, as shown in FIG. 3B, the user holds the mobile phone terminaldevice 100 in a state in which the fingers f2 to f5 other than the thumbf1 and the palm of the hand cover the surface of the display panel 121of the mobile phone terminal device 100.

When the fact that the state shown in FIG. 3B is reached is detected bythe touch-panel unit 130 in the mobile phone terminal device 100, thecontrol unit 110 of the mobile phone terminal device 100 controlsdisplay on the display panel 211 of the display device 200 on the basisof the motion detected by the motion sensor unit 108. The determinationthat the state shown in FIG. 3B is reached is performed, for example, onthe basis of the detection of the area of the surface of the displaypanel 121 that is covered on the touch-panel unit 130.

In the state shown in FIG. 3B, the display panel 121 of the mobile phoneterminal device 100 does not perform display, and the touch-panel unit130 performs only the touch detection. Specific examples of thesecontrol states will be described later. In the following description, astate in which a touch operation shown in FIG. 3A is performed will bereferred to as an ordinary touch input mode, and a state in which anoperation is performed by motion with the surface of the display panel121 being covered with fingers as shown in FIG. 3B will be referred toas a motion input mode.

[1-3. Control Process Based on Operation]

Next, specific control processes through which operations shown in FIG.3 are realized will be described.

FIG. 4 is a flowchart illustrating a control process performed by thecontrol unit 110 of the mobile phone terminal device 100. Initially, thecontrol unit 110 determines whether or not there is a change from thetouch state detected by the touch-panel unit 130 to a state in which thedisplay panel 121 is covered with the hand (step S11). The change of thestate in which the display panel 121 is covered with the hand isperformed by the touch-panel unit 130 by detecting which degree of areaof the surface of the display panel 121 is covered by an object, such asfingers. Specifically, for example, when the touch-panel unit 130detects that 50% or more of the surface of the display panel 121 iscovered, the control unit 110 determines that the surface of the displaypanel 121 is covered with the hand. 50% as a threshold value forperforming the determination is an example, and another value may beset. Furthermore, the touch-panel unit 130 may detect another touchstate other than the touched area, and may detect that the touch-panelunit 130 is covered with the hand.

Then, when it is determined in step S11 that there is a change in thestate in which the display panel 121 is covered by the hand, the controlunit 110 determines whether or not the present state is a state in whichthe touch-panel unit 130 is covered with the hand (step S12). When it isdetermined here that the touch-panel unit 130 is not covered with thehand, the control unit 110 starts the display on the display panel 121,and activates the touch detection in the touch-panel unit 130 (stepS13). In a case where the display panel 121 is a liquid-crystal displaypanel, the control unit 110 causes the backlight to be turned on insynchronization with the start of the display. Furthermore, the controlunit 110 deactivates the motion detection of the mobile phone terminaldevice 100 in the motion sensor unit 108 (step S14). After the processof step S14 is performed, the control unit 110 returns to thedetermination of step S11.

Furthermore, when it is determined in step S12 that the touch-panel unit130 is covered with the hand, the control unit 110 completes the displayon the display panel 121, and deactivates the touch detection in thetouch-panel unit 130 (step S15). In a case where the display panel 121is a liquid-crystal display panel, the control unit 110 causes thebacklight to be turned off in synchronization with the ending of thedisplay. Furthermore, the control unit 110 activates the motiondetection of the mobile phone terminal device 100 in the motion sensorunit 108 (step S16). After the process of step S16 is performed, thecontrol unit 110 returns to the determination of step S11.

Then, when it is determined in step S11 that there is no change in thestate in which the display panel 121 is covered with the hand, thecontrol unit 110 determines whether or not the present state is a statein which the touch-panel unit 130 is covered with the hand (step S17).When it is determined here that the touch-panel unit 130 is not coveredwith the hand, the control unit 110 performs a control process of adisplay video based on the touch detection in the touch-panel unit 130(step S18). After the process of step S18 is performed, the control unit110 returns to the determination of step S11.

Furthermore, when it is determined in step S17 that the touch-panel unit130 is covered with the hand, the control unit 110 determines whether ornot the motion sensor unit 108 has detected the motion of the mobilephone terminal device 100 (step S19). When it is determined here thatthe motion sensor unit 108 has detected the motion of the mobile phoneterminal device 100, the control unit 110 transmits an instructioncorresponding to the detected motion to the display device 200 (stepS20). Then, after the process of step S20 is performed and when themotion sensor unit 108 does not detect the motion in step S19, thecontrol unit 110 returns to the determination of step S11.

FIG. 5 illustrates a flow of the process of each unit in the mobilephone terminal device 100. At the initial timing of the process of FIG.5, the user is in a state of holding the mobile phone terminal device100 as shown in FIG. 3A. In this state, the mobile phone terminal device100 is operating in the ordinary touch input mode. During the operationin the ordinary touch input mode, when the touch-panel unit 130 detectsa touch operation (step S101), the coordinate report indicating thedetected touch coordinate position is transmitted from the touch-panelunit 130 to the control unit 110 (step S102). The control unit 110determines which place of the display image was touched on the basis ofthe transmitted coordinate report, and performs a process on the basisof the determination. The description of the processing performed by thecontrol unit in the ordinary touch input mode is omitted.

After that, the user changes the state of having the mobile phoneterminal device 100, and a state in which the display panel 121 iscovered with the palm of the hand is reached, as shown in FIG. 3B. Atthis time, the touch-panel unit 130 detects that the palm of the hand istouched (step S103). When the touch-panel unit 130 detects that the palmof the hand has been touched, the touch-panel unit 130 deactivates theordinary coordinate detection of the touch operation (step S104). Then,the touch-panel unit 130 transmits the detection report of the hand'spalm touch to the control unit 110 (step S105). The control unit 110receiving the detection report of the hand's palm touch switches off thedisplay of the display panel 121 (step S106), and changes the operationmode to a motion input mode.

When the motion input mode is entered, the control unit 110 sends aninstruction for activating an acceleration sensor and a gyro sensor tothe motion sensor unit 108 (step S201). The motion sensor unit 108 uponreceiving this instruction, on the basis of the output of the gyro unit108 a and the acceleration sensor 108 b, detects the inclination and themotion of the mobile phone terminal device 100 (step S202), andtransmits the report of the detection result to the control unit 110(step S203).

The control unit 110 upon receiving the report of the inclination andthe motion of the mobile phone terminal device 100 converts the gestureresulting from the motion or the inclination indicated by the receivedreport into a key signal that instructs performance of an operation(step S204). Then, the control unit 110 supplies the converted keysignal to the short-distance wireless communication processing unit 107(step S205). The short-distance wireless communication processing unit107 to which this key signal has been transmitted generates a controlsignal for wireless transmission corresponding to the key signal, andwirelessly transmits this control signal to the display device 200 (stepS206).

The display device 200 upon receiving this control signal judges thecontent of the control signal, and performs a process for the videodisplayed on the display panel 211. For example, the control unit 203 ofthe display device 200 performs a process for changing the position ofthe pointer that is displayed in such a manner as to be superimposed onthe video.

The process of FIG. 5 is an example in which, during the motion inputmode, a control signal is transmitted from the mobile phone terminaldevice 100 to the display device 200, and the video is changed in thedisplay device 200 on the basis of the motion and the inclination thatare detected by the motion sensor unit 108.

In contrast, during the motion input mode, the video to be output by themobile phone terminal device 100 may be changed on the basis of themotion and the inclination that are detected by the motion sensor unit108.

FIG. 6 illustrates a flow of the process in this case.

The processing from step S101 to step S106, and the processing from stepS201 to step S203 in FIG. 6 are the same as the processing shown in FIG.5. Then, the control unit 110 upon receiving the report of the detectionresults of the inclination and the motion of the mobile phone terminaldevice 100 in step S203 generates an instruction based on the report(step S211). The instruction generated by the control unit 110 is sentto the display processing unit 120 (step S212). For example, the controlunit 110 instructs the change of the position of the pointer in thevideo. The video processing unit 120 upon receiving this instructionchanges the position of the pointer of the video being generated. Then,the video processing unit 120 generates a video in which the pointerposition has been changed (step S213), and supplies the video after thepointer position change to the short-distance wireless communicationprocessing unit 107 (step S214).

Then, the short-distance wireless communication processing unit 107modulates the supplied video data to that for wireless transmission, andwirelessly transmits the data to the display device 200 (step S215). Thedisplay device 200 performs a process for displaying the received videodata.

In FIG. 6, only the transmission of the video data after the pointerposition change is shown as steps S214 and S215. In a state in which thewireless connection between the mobile phone terminal device 100 and thedisplay device 200 is completed, the wireless transmission of the videodata is constantly performed.

FIG. 7 illustrates processing in which the control unit 110 receivingthe report of the detection results of the inclination and the motionfrom the motion sensor unit 108 generates an instruction for performingprocessing regarding a video. The flowchart of FIG. 7 is processingperformed in step S204 of FIG. 5 and in step S211 of FIG. 6.

First, the control unit 110 determines whether or not there is anoperation of an up key (operation key 144: FIG. 2) (step S21). When itis determined in this determination that there is an operation of the upkey, the control unit 110 increases by one step the movement speed ofthe pointer in the motion input mode (step S22).

Then, when it is determined in step S21 that there is no operation ofthe up key, the control unit 110 determines whether or not there is anoperation of a down key (operation key 145: FIG. 2) (step S22). When itis determined in this determination that there is an operation of thedown key, the control unit 110 decreases by one step the movement speedof the pointer in the motion input mode (step S24).

Then, when it is determined in the determination of step S22 that thereis no operation of the down key, the control unit 110 determines whetheror not a motion has been detected on the basis of the report from themotion sensor unit 108 (step S25). When it is determined in thisdetermination that a motion has been detected, the control unit 110determines whether or not a motion of being swung, in which theorientation of the mobile phone terminal device 100 is changed, has beendetected (step S26). When a motion of being swung, in which theorientation of the mobile phone terminal device 100 has been changed,has been detected in this determination, the control unit 110 generatesan instruction corresponding to a tap operation (step S27). Then, when amotion of being swung has not been detected, the control unit 110determines whether or not the detected speed of the motion is a movementspeed smaller than or equal to a predetermined threshold value (stepS28).

When a motion in which the movement speed is smaller than or equal tothe threshold value has been detected in step S28, an instruction formoving the position of the pointer in the video is generated on thebasis of the movement distance of the motion detected by the motionsensor unit 108 (step S29).

Furthermore, when a motion in which the movement speed is greater thanthe threshold value has been detected in step S28, the control unit 110generates an instruction for scrolling the screen in the moved direction(step S30).

Then, after processes of steps S22, S24, S27, and S29 are performed, thecontrol unit 110 returns to the determination of step S21. Furthermore,when it is determined in step S25 that no motion has been detected, thecontrol unit 110 returns to the determination of step S21.

[1-4. Examples of Specific Operations and Displays]

Next, specific change examples of video, which are performed in theprocessing shown in the flowchart of FIG. 7, will be described withreference to FIGS. 8 (A-C) to 10 (A and B).

FIGS. 8A-8C illustrate examples in which the position of the pointer inthe video displayed by the display device 200 is operated by the motionof the mobile phone terminal device 100. All the states shown in FIGS.8A, B, and C are states in which the display panel 211 of the displaydevice 200 displays the video output from the mobile phone terminaldevice 100. Furthermore, the states shown in FIGS. 8A, B, and C arestates in which the display panel 121 of the mobile phone terminaldevice 100 is covered with the hand of the user. In this state, themobile phone terminal device 100 is operating in the motion input mode.

The example of FIG. 8A is an example in which the display panel 211 ofthe display device 200 displays the video of the home page of a certainsite. The display video of this example contains a status bar 221, asearch character input place 222, and a news display place 223. Thestatus bar 221 displays the state of the mobile phone terminal device100. Furthermore, a pointer P1 pointing to a specific place in the videois displayed in nearly the center of the video.

Then, at such a display state, by moving the mobile phone terminaldevice 100 by the user, the display position of the pointer P1 ischanged. For example, as shown in FIG. 8B, when the user performs amotion m1 of moving the mobile phone terminal device 100 to the upperright, in synchronization with the motion m1 to the upper right, thepointer P1 in nearly the center is changed to a pointer P2 at a positionin the upper right. The amount of movement of the pointer P1 to P2 isproportional to the amount of movement of the motion m1 of the mobilephone terminal device 100.

The relationship between the amount that the mobile phone terminaldevice 100 has moved and the amount of motion of the pointer on thescreen can be adjusted by the user by operating the up key and the downkey. That is, as described in the flowchart of FIG. 7, when the up keyis operated, the movement speed of the pointer increases, and when thedown key is operated, the movement speed of the pointer decreases.

FIG. 8C illustrates an example in which the movement speed is adjustedto be lower than that in the state of FIG. 8B. As shown in FIG. 8C, whenthe user performs substantially the same motion m1′ as in the state ofFIG. 8B in which the mobile phone terminal device 100 is moved to theupper right, the pointer P1 in nearly the center is changed to a pointerP3, in which the amount of movement is smaller than that of the pointerP2 of FIG. 8B.

The process for superimposing the pointer in the display video may beperformed by the display device 200 and performed by the mobile phoneterminal device 100. The flow of the process, shown in FIG. 5, is anexample in which the display device 200 performs a process forsuperimposing the pointer in the display video, and the flow of theprocess, shown in FIG. 6, is an example in which the mobile phoneterminal device 100 performs a process for superimposing the pointer inthe display video.

The process for instructing the movement of the pointer with the motionof the mobile phone terminal device 100, which is shown in FIG. 8, is acase in which the user moves the mobile phone terminal device 100 at acomparatively slow speed. On the other hand, in a case where the usermoves the mobile phone terminal device 100 at a comparatively fastspeed, an operation differing from the movement of the pointer isperformed. The speed determination is performed in step S26 of theflowchart of FIG. 7. The threshold value of the speed for determinationmay be made to be able to be changed by user setting.

FIGS. 9A and 9B illustrate other examples showing states in which themovement speed is changed by the operation of the up key and the downkey.

FIG. 9A illustrates a case in which an operation of the up key isperformed. In this case, when there is a motion m11 of a fixed distancein the horizontal direction with respect to the mobile phone terminaldevice 100, a pointer P11 on the screen is changed to a pointer P12 at aposition moved more than the position before the up key was operated.The speed at which the display position of the pointer moves increasesin response to the number of times the up key is pressed.

FIG. 9B illustrates an example in which the down key is operated in thesame situation as that of FIG. 9A. In this case, when there is a motionm11 of a fixed distance in the horizontal direction with respect to themobile phone terminal device 100, the pointer P11 on the screen ischanged to a pointer P13 at a position before the down key was operated.The speed at which the display position of the pointer moves decreasesin response to the number of times the up key is pressed.

As described above, the movement speed can be adjusted by operating theup key and the down key, making it possible to easily perform anoperation of adjusting the pointer (cursor) on the screen to a fine iconon the screen. The operation keys 144 and 145 serving as an up key and adown key are keys arranged on the side surface of the housing, as shownin FIG. 2. It is possible for the user to satisfactorily performoperations by using fingers comparatively easily in a state in which thedisplay panel is covered with the hand h of the user.

FIGS. 10A and 10B illustrate examples of the motion of the mobile phoneterminal device 100 in a case where the user moves the mobile phoneterminal device 100 at a comparatively fast speed.

FIG. 10A illustrates an example when the user performs a motion m2 ofinclining the mobile phone terminal device 100 at a comparatively fastspeed. At this time, upon the detection of a comparatively high-speedmotion m2 involving the inclination in the motion sensor unit 108 of themobile phone terminal device 100, the control unit 110 performs aninstruction for a tap operation. A tap operation is an operation ofselecting a specific place on the screen. For example, as shown in FIG.8B, the user performs a tap operation in a state in which the pointer P2points to a search character input place 222. Consequently, thecharacter input place is selected, and the display panel 211 displays ascreen for performing character input.

FIG. 10B illustrates examples of cases where the user moves the mobilephone terminal device 100 up and down or side by side at a comparativelyfast speed. For example, when the motion sensor unit 108 detects amotion m3 at which the mobile phone terminal device 100 is moved upwardat a comparatively fast speed, the control unit 110 performs an upwardscroll instruction on the screen. As a result of performing this upwardscroll instruction, the video displayed on the display panel 211 of thedisplay device 200 is scrolled in an upward direction.

In a similar manner, when the user performs a motion m4 in thehorizontal direction or a motion m5 in the downward direction, thescroll in the horizontal direction or the scroll in the downwarddirection of the video displayed on the display panel 211 of the displaydevice 200 is performed.

By changing the way of moving the mobile phone terminal device 100 bythe user in a state in which the display panel 121 of the mobile phoneterminal device 100 is covered with fingers and the palm of the hand asdescribed above, it is possible to perform an operation of moving thepointer, a tap operation, an operation of scrolling the entire screen.

Furthermore, when the movement of the pointer is to be performed,adjustment of a speed is possible by operating the up key and the downkey. Thus, an operation with high accuracy in which the intent of theuser is reflected becomes possible.

Furthermore, switching from the ordinary touch input mode to the motioninput mode is performed by only covering the display panel 121 of themobile phone terminal device 100 with fingers and the palm of the hand.Consequently, mode switching can be performed easily by only passing themobile phone terminal device 100 from one hand to the other by the user.

<2. Second Embodiment>

[2-1. Control process based on operation]

An example of a second embodiment will be described with reference toFIGS. 11 to 12 (A and B). In FIGS. 11 to 12 (A and B), the samecomponents as those in FIGS. 1 to 10 described earlier in the firstembodiment are designated with the same reference numerals, and detaileddescription thereof is omitted.

The example of the second embodiment differs from the first embodimentin the control state in which the mobile phone terminal device 100 andthe display device 200 are connected to each other, and the mobile phoneterminal device 100 is placed in a motion input mode. For theconfiguration of the mobile phone terminal device 100 and the displaydevice 200, the configuration shown FIGS. 1 and 2, which are describedearlier in the first embodiment, is applied.

The flowchart of FIG. 11 illustrates processing performed by the controlunit 110 during the motion input mode in the second embodiment. The modeshifts to a motion input mode when, as described in the firstembodiment, a fixed area or more of the display panel 121 of the mobilephone terminal device 100 is covered with fingers and the palm of thehand.

As shown in FIG. 11, when the mode goes to the motion input mode, thecontrol unit 110 determines whether or not the mobile phone terminaldevice 100 is in a horizontal state on the basis of the detection stateof the motion sensor unit 108 (step S41). When it is determined herethat the mobile phone terminal device 100 is not in a horizontal state,the control unit 110 performs an operation instruction based on thedetection output of the motion sensor unit 108 (step S42). For theprocess for performing an operation instruction based on the detectionoutput of the motion sensor unit 108, for example, the process shown inthe flowchart of FIG. 7 can be applied. Furthermore, at this time, thethree operation keys 141, 142, and 143 arranged on the surface of themobile phone terminal device 100 are assumed to be keys havingpre-assigned functions.

Then, when it is determined in step S41 that the mobile phone terminaldevice 100 is nearly in a horizontal state, the control unit 110 assignsthe three operation keys 141, 142, and 143 arranged on the surface ofthe mobile phone terminal device 100 to right and left click operationsand a scroll operation (step S43). Then, the position instructed by thecontrol unit 110 is changed in response to the motion of the mobilephone terminal device 100, which is detected by the motion sensor unit108 (step S44).

After the processes of steps S42 and S44 are performed, the control unit110 returns to the determination of step S41.

2-2. Examples of Specific Operations

FIGS. 12A and 12B illustrate a state (FIG. 12A) in which the surface ofthe display panel 121 is covered with fingers and the palm of the handin a state in which the user holds the mobile phone terminal device 100,and a state (FIG. 12B) in which the display panel 121 of the mobilephone terminal device 100 that is placed nearly horizontally is coveredwith fingers and the palm of the hand.

As shown in FIG. 12A, in a state in which the mobile phone terminaldevice 100 is held by the hand h of the user, the display panel 121 isin a nearly upright state. At this time, similarly to the case that hasalready been described in the example of the first embodiment, anoperation instruction regarding the displayed video is performed inresponse to the motion and the inclination detected by the motion sensorunit 108.

Then, as shown in FIG. 12B, in a state in which the mobile phoneterminal device 100 is placed horizontally on the table 300 and user'sfingers and the palm of the hand are placed on the display panel 121,the three keys 141, 142, and 143 are assigned to right and left clickoperation keys and a scroll operation key. Specifically, the key 141 isdesignated as a right click operation key, the key 142 is designated asa scroll operation key, and the key 143 is designated as a left clickoperation key. At this time, the orientation in which the mobile phoneterminal device 100 is arranged is an orientation in which the keys 141,142, and 143 are oriented upward.

As a result of the keys 141 to 143 being assigned as described above,the mobile phone terminal device 100 performs the same operation as thatof a pointing device, such as a mouse. That is, as shown in FIG. 12B, ina state in which fingers and the palm of the hand are placed on thedisplay panel 121, the user performs an operation of pressing the keys141 to 143 with the tip ends of the three fingers and also, the positionof the mobile phone terminal device 100 is moved, making it possible toperform the same instruction as that by the mouse.

In the state shown in FIG. 12B, fingers and the palm of the hand becomeslightly away from the surface of the display panel 121. However, sincethe touch-panel unit 130 detects an approach of fingers even in an awaystate, the touch-panel unit 130 can detect that the mobile phoneterminal device 100 is in a state of being covered.

By performing operations with the mobile phone terminal device 100 beingplaced horizontally as shown in FIG. 12B, the pointer in the videodisplayed on the display panel 211 of the display device 200 can bemoved, or a click operation or a scroll operation can be performed.Thus, operations like a mouse are made possible.

When the assignment of the three keys 141 to 143 are to be changed, thetwo keys 144 and 145 (up key and down key) arranged on the side surfaceof the mobile phone terminal device 100 may also be changed to keyshaving more ease of use.

Furthermore, in the flowchart of FIG. 11, even when the mobile phoneterminal device 100 is not in a horizontal state shown in FIG. 12A,operations are performed by assuming the mode to be a motion input mode.Alternatively, only when the mobile phone terminal device 100 is in ahorizontal state shown in FIG. 12B, operations are performed by assumingthe mode to be a motion input mode.

Furthermore, in the example of the second embodiment, in the same way asin the example of the first embodiment, a case in which the mobile phoneterminal device 100 and the display device 200 are wirelessly connectedto each other has been described. In contrast, the device of anotherparty that performs the instruction from the mobile phone terminaldevice 100 may be a device other than the display device. For example,the mobile phone terminal device 100 may wirelessly transmit aninstruction based on a motion or a key operation to a personal computerdevice. As a result of the above, the mobile phone terminal device 100functions as a mouse for a personal computer device.

<3. Modifications>

In the above-described embodiments, examples of cases in which thedisplay device 200 displays the same video as the video displayed by thedisplay panel 121 of the mobile phone terminal device 100 have beendescribed. In contrast, the present disclosure may be applied to a casein which the display device 200 displays a video different from thevideo displayed by the display panel 121 of the mobile phone terminaldevice 100.

For example, as shown in FIG. 13A, a state is assumed in which thedisplay panel 211 of the display device 200 displays a video 224 ofcontent downloaded by the mobile phone terminal device 100, and thedisplay panel 121 of the mobile phone terminal device 100 displays anelectronic mail creation screen.

The video 224 displayed by the display device 200 is attached with aplayback button 225, a playback position display 226, and the like.Examples of screens displayed by the mobile phone terminal device 100include an input screen 121 a and a keyboard screen 121 b.

In such a state, for example, it is assumed that the user changes theway of holding the mobile phone terminal device 100, and as shown inFIG. 13B, fingers and the palm of the hand are positioned on the surfaceof the display panel 121 of the mobile phone terminal device 100. Atthis time, in the mobile phone terminal device 100, a motion input modeis set, the display of the display panel 121 is switched off and also,the video output by the mobile phone terminal device 100 is switchedfrom the playback video of the video content to an electronic mailcreation video.

As a result of the above, the display panel 211 of the display device200 displays an electronic mail input screen 228, and a keyboard screen229. Then, by moving the mobile phone terminal device 100, it becomespossible for the user to perform a selection operation for characterinput or the like.

In the example of FIG. 13B, in the display device 200, the display ofthe video content is completely erased, and only the electronic mailcreation screen is displayed. In contrast, the display of the videocontent and the display of the electronic mail creation screen may beperformed at the same time.

Furthermore, in each of the above-described embodiments, an example ofthe case in which the mobile phone terminal device 100 configured as asmartphone and the display device 200 configured as a televisionreceiver are connected to each other has been described. In contrast,the present disclosure may be applied to a case in which anotherterminal apparatus and another display device are connected to eachother.

Furthermore, in each of the above-described embodiments, the terminalapparatus and the display device are wirelessly connected to each other,and the display device displays a video. In contrast, the sameprocessing may be performed in a case where a terminal apparatus and adisplay device are connected to each other through a wire cable, and thedisplay device displays a video transmitted from the terminal apparatus.

In the case of this connection by a wire cable, for example, processingperformed by the short-distance wireless communication processing unit107 shown in FIG. 5 or 6 becomes processing performed by theinput/output processing unit 160. It is possible for the input/outputprocessing unit 160 to perform processing in a case where communicationin the terminal part 170 to which a cable of the HDMI standard isconnected is controlled, and a display device to which the cable of theHDMI standard is connected performs display.

Furthermore, programs (software) for performing control processing,described in the flowcharts of FIGS. 4, 7 and 11, may be generated, andthe programs may be stored on a recording medium. A terminal apparatus,in which the programs are installed by preparing the programs stored onthe recording medium, becomes a terminal apparatus that performs theprocessing of the present disclosure.

The configurations and processes disclosed in the claims of the presentdisclosure are not limited to the above-described embodiments. It shouldbe understood of course by those skilled in the art that variousmodifications, combinations, and other exemplary embodiments may be madeaccording to design and/or other factors insofar as they come within thescope of the claims or the equivalents thereof.

The present disclosure can take the following configurations.

(1) A terminal apparatus including

a display panel arranged on a housing;

a display processing unit that processes an image displayed on thedisplay panel;

an output unit that externally outputs the image processed by thedisplay processing unit or an instruction regarding the image;

a touch-panel that detects a contact or approach of an object with ortoward a surface of the display panel;

a motion sensor that detects a motion of the housing; and

a control unit having circuitry that when the touch-panel detects thecontact or approach of the object having a greater size than or equal toa predetermined size, controls the output unit to output an image or aninstruction corresponding to the motion detected by the motion sensor.

(2) The terminal apparatus of (1), wherein when the control unit detectsthat the contact or approach of the object of greater than or equal tothe predetermined size, the control unit deactivates the touch-panelfrom detecting further contact or approach of the object.

(3) The terminal apparatus of 1, wherein when the control unit detectsthat the contact or approach of the object of greater than or equal tothe predetermined size, the control unit activates the motion sensor todetect the motion of the housing.

(4) The terminal apparatus of 3, wherein the control unit identifies amovement of the housing that is not a swinging movement as correspondingto a tap operation that selects a specific location on the displaypanel.

(5) The terminal apparatus of 3, wherein the control unit identifies aswing of the housing as being a scrolling operation on the display panelwhen a movement speed of the housing is not smaller than or equal to athreshold value.

(6) The terminal apparatus of 3, wherein the control unit identifies aswing of the housing as being a request to move a pointer position onthe display panel when a movement speed of the housing is smaller thanor equal to a threshold value.

(7) The terminal apparatus of 6, wherein the control unit causes thepointer position to move by an amount that corresponds to a movementamount of the housing.

(8) The terminal apparatus of 7, wherein

after the pointer position has been moved, the control unit determinesthat an operation other than movement of the pointer is performed inresponse to the housing being moved faster than the threshold value.

(9) The terminal apparatus of 1, wherein

the control unit responds to an inclination movement of the housing byperforming a tap operation that selects a specific location on thedisplay panel.

(10) The terminal apparatus of 1, wherein

the control unit responds to an up-and-down or a side-to-side movementof the housing by performing a scrolling operation on the display panel.

(11) The terminal apparatus of 1, further comprising:

a plurality of keys, wherein

when the control unit determines that a user's palm is placed on thetouch-panel, actuation of the plurality of keys is recognized as apointing device.

(12) The terminal apparatus of 11, wherein

the control unit recognizes actuation of a predetermined key as aleft-click operation.

(13) The terminal apparatus of 11, wherein

the control unit recognizes actuation of a predetermined key as aright-click operation.

(14) The terminal apparatus of 11, wherein

the control unit recognizes actuation of a predetermined key as a scrolloperation.

(15) The terminal apparatus of 1, wherein

the control unit is configured to erase the image of the display paneland insert an image of an electronic mail creation screen.

(16) The terminal apparatus of 15, wherein

the control unit outputs the instruction to create an e-mail message inresponse to motions detected by the motion sensor.

(17) A display method including

displaying an image on a display panel arranged on a housing;

processing with a display processing unit the image displayed on thedisplay panel;

externally outputting the image processed by the display processing unitor an instruction regarding the image;

detecting on a touch-panel a contact or approach of an object with ortoward a surface of the display panel;

detecting with a motion sensor a motion of the housing; and

performing with a control unit having circuitry when the touch-paneldetects the contact or approach of the object having a greater size thanor equal to a predetermined size, a control of the output unit to outputan image or an instruction corresponding to the motion detected by themotion sensor.

(18) The method of 17, wherein

the detecting includes detecting the contact or approach of the objectof greater than or equal to the predetermined size and deactivating thetouch-panel from detecting further contact or approach of the object.

(19) The method of 17, wherein

the detecting includes detecting the contact or approach of the objectof greater than or equal to the predetermined size and activating themotion sensor to detect the motion of the housing.

(20) A non-transitory computer readable storage device having computerreadable instructions that when executed by processing circuitry causethe processing circuitry to implement a display method, the methodincluding

displaying an image on a display panel arranged on a housing;

processing with a display processing unit the image displayed on thedisplay panel;

externally outputting the image processed by the display processing unitor an instruction regarding the image;

detecting on a touch-panel a contact or approach of an object with ortoward a surface of the display panel;

detecting with a motion sensor a motion of the housing; and

performing with a control unit having circuitry when the touch-paneldetects the contact or approach of the object having a greater size thanor equal to a predetermined size, a control of the output unit to outputan image or an instruction corresponding to the motion detected by themotion sensor.

REFERENCE SIGNS LIST

100 . . . mobile phone terminal device, 101 . . . antenna, 102 . . .wireless communication processing unit, 103 . . . audio processing unit,104 . . . speaker, 105 . . . microphone, 106 . . . antenna, 107 . . .short-distance wireless communication processing unit, 108 . . . motionsensor unit, 108 a . . . gyro unit, 108 b . . . acceleration sensorunit, 110 . . . control unit, 120 . . . display processing unit, 121 . .. display panel, 130 . . . touch-panel unit, 141 to 145 . . . operationkey, 160 . . . input/output processing unit, 170 . . . terminal part,200 . . . display device, 201 . . . terminal part, 202 . . . switchingunit, 203 . . . control unit, 204 . . . tuner, 205 . . . antenna, 206 .. . short-distance wireless communication processing unit, 210 . . .display unit, 211 . . . display panel, and 300 . . . table

The invention claimed is:
 1. A terminal apparatus comprising: a displaypanel arranged on a housing of the terminal apparatus; and circuitryconfigured to process an image displayed on the display panel, output,to an external device, at least one of the processed image by thecircuitry or an instruction regarding the processed image using wirelesscommunication, detect a motion of the housing, and output, to theexternal device, an updated instruction corresponding to the motiondetected by the circuitry, the updated instruction including aninstruction to display an electronic mail (e-mail) application, whereinthe circuitry causes a scrolling operation instruction of the processedimage on the external device or a moving operation instruction of apointer position on the external device based on whether or not amovement speed of the housing is smaller than or equal to a thresholdvalue, wherein the circuitry is further configured to adjust arelationship between a movement amount of the housing and a movementamount of the pointer position during the moving operation through anoperation of one or more keys on the housing of the display panel, andwherein at least a portion of the display of the external device isswitched from the image application to the e-mail application.
 2. Theterminal apparatus of claim 1, wherein the circuitry responds to aninclination movement of the housing by performing a tap operation thatselects a specific location on the display panel.
 3. The terminalapparatus of claim 1, wherein the circuitry responds to an up-and-downor a side-to-side movement of the housing by performing a scrollingoperation on the display panel.
 4. The terminal apparatus of claim 1,wherein the circuitry is further configured to perform one or morecharacter input selection operations, on the external device, based onthe detected motion of the housing.
 5. The terminal apparatus of claim1, wherein the updated instruction includes an instruction to erase theprocessed image and update the processed image with the e-mailapplication, the e-mail application including an e-mail creationmessage.
 6. The terminal apparatus of claim 1, wherein the circuitryfurther includes a touch panel configured to detect a contact orapproach of an object with or toward a surface of the display panel. 7.The terminal apparatus of claim 6, wherein the circuitry is furtherconfigured to output the processed image or the instructioncorresponding to the motion detected by the circuitry when the touchpanel detects the contact or approach of the object having a greatersize than or equal to a predetermined size.
 8. The terminal apparatus ofclaim 6, wherein the circuitry deactivates detecting further contact orapproach of the object when the circuitry detects the contact orapproach of the object of greater than or equal to a predetermined size.9. The terminal apparatus of claim 6, wherein the circuitry detects themotion of the housing when the circuitry detects the contact or approachof the object of greater than or equal to a predetermined size.
 10. Theterminal apparatus of claim 9, wherein the circuitry identifies amovement of the housing that is not a swinging movement as correspondingto a tap operation that selects a specific location on the displaypanel.
 11. The terminal apparatus of claim 10, wherein the circuitrycauses the pointer position to move by an amount that corresponds to amovement amount of the housing.
 12. The terminal apparatus of claim 11,wherein after the pointer position has been moved, the circuitrydetermines that an operation other than movement of the pointer isperformed in response to the housing being moved faster than thethreshold value.
 13. The terminal apparatus of claim 1, furthercomprising: a plurality of keys, wherein when the circuitry determinesthat a user's palm is placed on the touch-panel, actuation of theplurality of keys is recognized as a pointing device.
 14. The terminalapparatus of claim 13, wherein the circuitry recognizes actuation of apredetermined key as a left-click operation.
 15. The terminal apparatusof claim 13, wherein the circuitry recognizes actuation of apredetermined key as a right-click operation.
 16. The terminal apparatusof claim 13, wherein the circuitry recognizes actuation of apredetermined key as a scroll operation.
 17. A display methodcomprising: displaying an image on a display panel arranged on a housingof a terminal apparatus; processing, with circuitry, the image displayedon the display panel; outputting, to an external device, the processedimage by the circuitry or an instruction regarding the processed image;detecting a motion of the housing; outputting, to the external device,an updated instruction corresponding to the motion detected by thecircuitry, the updated instruction including an instruction to displayan electronic mail (e-mail) application; causing, with the circuitry, ascrolling operation instruction of the processed image on the externaldevice or a moving operation instruction of a pointer position on theexternal device based on whether or not a movement speed of the housingis smaller than or equal to a threshold value; adjusting, with thecircuitry, a relationship between a movement amount of the housing and amovement amount of the pointer position during the moving operationthrough an operation of one or more keys on the housing of the displaypanel; and switching at least a portion of the display of the externaldevice from the image application to the e-mail application.
 18. Themethod of claim 17, further comprising: deactivating a touch panel fromfurther detecting when the detecting includes detecting a contact orapproach of an object with or toward a surface of the display panel ofgreater than or equal to a predetermined size.
 19. The method of claim17, wherein the motion of a housing is detected when a contact orapproach of an object of greater than or equal to a predetermined sizeis detected.
 20. A non-transitory computer readable storage devicehaving computer readable instructions that when executed by processingcircuitry cause the processing circuitry to perform a display methodcomprising: displaying an image on a display panel arranged on a housingof a terminal apparatus; processing the image displayed on the displaypanel; outputting, to an external device, the processed image or aninstruction regarding the processed image; detecting a motion of thehousing; outputting, to the external device, an updated instructioncorresponding to the motion detected, the updated instruction includingan instruction to display an electronic mail (e-mail) application;causing a scrolling operation instruction of an image on the externaldevice or a moving operation instruction of a pointer position on theexternal device based on whether or not a movement speed of the housingis smaller than or equal to a threshold value; adjusting a relationshipbetween a movement amount of the housing and a movement amount of thepointer position during the moving operation through an operation of oneor more keys on the housing of the display panel; and switching at leasta portion of the display of the external device from the imageapplication to the e-mail application.